Polynomial fits using fit_pandas_GUI()

You can try this notebook live by lauching it in Binder.This can take a while to launch, be patient. .

First we import pandas and pandas_GUI and then load some data to fit.

import pandas as pd
from pandas_GUI import *
df = pd.read_csv('DataSets/IRoscbaseline.csv.gz')

Then make a quick plot using plot_pandas_GUI() to see what the data looks like. See the Pandas GUI Website for examples and documentation on using the plotting GUI.

# CODE BLOCK generated using plot_pandas_GUI(). See https://github.com/JupyterPhysSciLab/jupyter_Pandas_GUI.
from plotly import graph_objects as go
Figure_1 = go.FigureWidget(layout_template="simple_white")
scat = go.Scatter(x = df['1/CM'], y = df['oscbase'],
        mode = 'lines', name = 'IR bad baseline',)
Figure_1.add_trace(scat)
Figure_1.update_xaxes(title= '1/cm', mirror = True)
Figure_1.update_yaxes(title= 'Absorbance', mirror = True)
Figure_1.update_layout(title = 'Figure_1', template = 'simple_white')
Figure_1.show()
# Force save widget states so that graph will still be
# available when notebook next opened in trusted state.
JPSLUtils.OTJS('Jupyter.actions.call("widgets:save-with-widgets");')

Figure 1: IR spectrum of ethane with an oscillating baseline.

Fitting the baseline to a polynomial

Looking at this spectrum it appears that the baseline oscillates through zero three time, so a good place to start is to try fitting it to a cubic polynomial.

Start by launching the fitting GUI by putting the command fit_pandas_GUI() in an empty cell and running the cell.

1. On the first tab

the data set and a name for the trace is set. See the screenshot below.

2. On the second tab

the default "none" was kept for the Error Type. See the screen shot below.

3. On the third tab

"PolynomialModel" was chosen from the popup menu and coefficients c4 - c7 were fixed at the default value of zero so terms of order higher than 3 were ignored. See the screenshot below.

4. On tab 4

the ranges of the data to fit to were chosen to ignore the peaks. Consecutive pairs of selected points starting with the lowest data index number define each range. Points can be deselected by holding down the ctrl key while clicking on a point.

The Extend fitted function plot box was also checked so that the fit is shown in the ignored regions.

See the screenshot below.

5. On the fifth tab

labels for the X and Y axis were input and the Display Mirror Axes box was checked.

6. On the last (sixth) tab

the final checks were done and then the 'Do Fit' button was clicked, closing the GUI and running the code in the cell below to perform the fit and display the results.

# CODE BLOCK generated using fit_pandas_GUI(). See https://github.com/JupyterPhysSciLab/jupyter_Pandas_GUI.
# Imports (no effect if already imported)
import numpy as np
import lmfit as lmfit
import round_using_error as rue
import copy as copy
from plotly import graph_objects as go
from IPython.display import HTML

# Define data and trace name
Xvals = df["1/CM"]
Yvals = df["oscbase"]
tracename = "IR with bad baseline"

# Define error (uncertainty)
Yerr = df["oscbase"]*0 + 1

# Define the fit model, initial guesses, and contraints
fitmod = lmfit.models.PolynomialModel()
fitmod.set_param_hint("c0", vary = True, value = 0.0)
fitmod.set_param_hint("c1", vary = True, value = 0.0)
fitmod.set_param_hint("c2", vary = True, value = 0.0)
fitmod.set_param_hint("c3", vary = True, value = 0.0)
fitmod.set_param_hint("c4", vary = False, value = 0.0)
fitmod.set_param_hint("c5", vary = False, value = 0.0)
fitmod.set_param_hint("c6", vary = False, value = 0.0)
fitmod.set_param_hint("c7", vary = False, value = 0.0)

# Define fit ranges
Yfiterr = copy.deepcopy(Yerr) # ranges not to fit = np.inf
Xfitdata = copy.deepcopy(Xvals) # ranges where fit not displayed = np.nan
Yfiterr[0:7] = np.inf
Xfitdata[0:7] = np.nan
Yfiterr[286:584] = np.inf
Xfitdata[286:584] = np.nan
Yfiterr[862:1291] = np.inf
Xfitdata[862:1291] = np.nan
Yfiterr[2311:3164] = np.inf
Xfitdata[2311:3164] = np.nan
Yfiterr[3571:3571] = np.inf
Xfitdata[3571:3571] = np.nan

# Do fit
Fit_1 = fitmod.fit(Yvals, x=Xvals, weights = 1/Yfiterr, scale_covar = True, nan_policy = "omit")

# Calculate residuals (data - fit) because lmfit
#  does not calculate for all points under all conditions
resid = []
for i in range(0,len(Fit_1.data)):
    resid.append(Fit_1.data[i]-Fit_1.best_fit[i])

# Plot Results
Fit_1_Figure = go.FigureWidget(layout_template="simple_white")
Fit_1_Figure.set_subplots(rows=2, cols=1, row_heights=[0.2,0.8], shared_xaxes=True)
scat = go.Scatter(y=resid,x=Xvals, mode="markers",name = "residuals")
Fit_1_Figure.update_yaxes(title = "Residuals", row=1, col=1, zeroline=True, zerolinecolor = "lightgrey", mirror = True)
Fit_1_Figure.update_xaxes(row=1, col=1, mirror = True)
Fit_1_Figure.add_trace(scat,col=1,row=1)
scat = go.Scatter(x=Xvals, y=Yvals, mode="markers", name=tracename)
Fit_1_Figure.add_trace(scat, col=1, row = 2)
Fit_1_Figure.update_yaxes(title = "Absorbance", row=2, col=1, mirror = True)
Fit_1_Figure.update_xaxes(title = "1/cm", row=2, col=1, mirror = True)
scat = go.Scatter(y=Fit_1.best_fit, x=Xvals, mode="lines", line_color = "black", name="extrapolated",line_dash="dash")
Fit_1_Figure.add_trace(scat, col=1, row=2)
scat = go.Scatter(y=Fit_1.best_fit,x=Xfitdata, mode="lines", name="fit", line_color = "black", line_dash="solid")
Fit_1_Figure.add_trace(scat,col=1,row=2)
Fit_1_Figure.show()

# Display best fit equation
fitstr = r'$fit = '
termcount = 0
for k in Fit_1.params.keys():
    pwr = int(str(k)[-1:])
    if Fit_1.params[k].vary:
        if termcount > 0:
            fitstr += ' + '
        fitstr += r'(\color{red}{'+rue.latex_rndwitherr(Fit_1.params[k].value,
                               Fit_1.params[k].stderr, errdig=1, lowmag=-3)+'})'
        if pwr == 1:
            fitstr += 'x'
        if pwr > 1:
            fitstr += 'x^'+str(pwr)
        termcount+=1
    else:
        if Fit_1.params[k].value!=0:
            if termcount > 0:
                fitstr += '+'
            fitstr += r'(\color{blue}{'+str(Fit_1.params[k].value)+'})'
            termcount +=1
            if pwr == 1:
                fitstr += 'x'
            if pwr > 1:
                fitstr += 'x^'+str(pwr)
fitstr+='$'
captionstr='<p>Use the command <code>Fit_1</code> as thelast line of a code cell for more details.</p>'
display(HTML(fitstr+captionstr))

$fit = (\color{red}{-0.4842\pm0.0007}) + (\color{red}{0.001222\pm0.000001})x + (\color{red}{(-6.734\pm0.008)\times 10^{-7}})x^2 + (\color{red}{(1.025\pm0.001)\times 10^{-10}})x^3$

Use the command Fit_1 as thelast line of a code cell for more details.

Figure 2: The results of the fit. Notice that the cubic polynomial was a very good choice. All the parameters appear to have three or four significant figures based on the uncertainties. Zooming in around zero on the residuals shows how well this flattens the baseline. Despite the good baseline correction the spectrometer or sampling methods really need fixing...

Plot of the spectrum with the baseline removed

As the residuals are just the spectrum with the fit subtracted from it we can use that as the corrected spectrum. If I were going to continue working with this spectrum I would add the corrected spectrum to the data frame. I am going to do that below, because it is easiest to make a plot from a data frame using plot_pandas_GUi().

df['corrected'] = resid

# CODE BLOCK generated using plot_pandas_GUI(). See https://github.com/JupyterPhysSciLab/jupyter_Pandas_GUI.
from plotly import graph_objects as go
Figure_3 = go.FigureWidget(layout_template="simple_white")
scat = go.Scatter(x = df['1/CM'], y = df['corrected'],
        mode = 'lines', name = 'Baseline Corrected IR',)
Figure_3.add_trace(scat)
Figure_3.update_xaxes(title= '1/cm', mirror = True)
Figure_3.update_yaxes(title= 'Absorbance', mirror = True)
Figure_3.update_layout(title = 'Figure_3', template = 'simple_white')
Figure_3.show()
# Force save widget states so that graph will still be
# available when notebook next opened in trusted state.
JPSLUtils.OTJS('Jupyter.actions.call("widgets:save-with-widgets");')

Figure 3: Plot of the baseline corrected spectrum that was added to the data frame from the residuals of the polynomial fit.

Learn More

In addition to trying it below if this is a live notebook, you can look at the other examples listed in the Pandas GUI website.

Try It

If you are running this notebook live in binder you can try it here by running the first cell to import the tools and load the data. Then run the cell below to create the fit GUI. Note: You may want to expand the collapsed instructions to learn more about each tab.

fit_pandas_GUI()